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EVALUATING TEACHING; 
BEYOND THE STUDENT QUESTIONNAIRE 
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Most discussions about evaluating teaching usually be- 
come discussions about the merits or faults of student 
questionnaires. Are students able to evaluate teaching? 
Should they? Do they take it seriously? Surely, given 
all the discussion and research in this area, there 
must be something beyond the student questionnaire for 
the assessment of teaching effectiveness. In this 
article we will briefly review one alternative - the 
teaching dossier or portfolio. This is a method of 
evaluation being currently discussed by various groups, 
including the CAUT Teaching Effectiveness Sub Committee 
of the Committee on Academic Affairs. 
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The evaluation of faculty has been the subject of con- 
siderable research and numerous methods have been de- 
veloped for the evaluation of teaching by students, by 
peers and by the instructor himself. The various ele- 
ments of a more comprehensive program of faculty evalu- 
ation exist, yet usually those elements are not assem- 
bled in a coherent and systematic manner. Evaluation 
Local 397 or 695 remains a most sensitive area for many faculty and 
evaluation decisions are a source of distress for per- 
sonnel committees and chairman. Even with the best of 
intentions significant career decisions are often still 
made on the basis of rumor, hearsay and innuendo; the 
faculty member feeling the passive victim of a process 
over which he has little or no control. 
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There is perhaps no one approach to the evaluation of 
faculty which will solve all of the problems. Dif- 
ficult and demanding decisions must still be made that 
will affect the futures of both individuals and in- 
stitutions; not everyone can get tenure, promotions 
and merit. The use of teaching portfolios to assess 
faculty performance may offer at least a partial ; 
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solution to two of the major obstacles in 
the evaluation of faculty; instead of a 
random and often haphazard process of doc- 
umentation and evaluation, a portfolio 
provides an approach that is both syste- 
matic and comprehensive; instead of fa- 
culty feeling like passive participants 
in an alienating evaluation process, the 
creation of a teaching portfolio actively 
involves the faculty member in the de- 
sign and implementation of his own eval- 
uation. 


WHAT IS A TEACHING PORTFOLIO? 


A teaching portfolio or dossier is a set 

of documents which collectively indicate 

a professor's commitment to teaching in 
terms of what he or she does to improve 
teaching and the results of those efforts. 
The portfolio approach to faculty evalua- 
tion may at first appear to be so detailed 
and even cumbersome as to be impossible to 
implement; faculty, it would seem, will be 
doing little else but collecting infor- 
mation for their portfolios. As the 
process becomes routine and as faculty 

gain the experience and skills needed 

for effective documentation, teaching 
portfolios can become a normal and 

accepted part of institutional life. 

For the most part, faculty appear 

more than willing to invest their 

time in an evaluation process over which 
they have some control. Finally, a 

full portfolio review need not be con- 
ducted every semester or even every 

year. Untenured faculty frequently face 
major reviews after their second or third 
year and again just before a tenure de- 
cision, more and more institutions are re- 
quiring three to five year reviews of 
tenured faculty. Carefully and slowly de- 
veloped over a number of semesters, a port-— 
folio can avoid the wasteful, disorganized 
and generally time-consuming scramble for 
documentation in the months and weeks just 
before review. The description offered in 
this newsletter of the use of portoflios in 
assessing faculty performance is only meant 
to be a model; it is expected and hoped 
that individual faculty, department chair- 
persons and administrators will be able 

to modify the model to meet their own unique 
needs and demands. 


WHAT GOES INTO A TEACHING PORTFOLIO? 


Following is a list of the types of information, 


and categories, that could be included in a 
portfolio: 


Reports from Self 


A. Statement describing current and recent teaching i 
sibilities and practices 

. Listing of course outlines, syllabl - 

. Statement of instructional goals, lists of general and 
special objectives. 

. Quizzes and examination Items perhaps keyed to In- - 
structional objectives and/or goals 

. Copies of handouts, worksheets, lab manuals, etc. 

. Lists of references, films other media used In In- . 
struction 

. Office hours and statement of avallability to students 

. Evidence about revising Instructional materials 
periodically and why 


8. Steps taken to evaluate and improve one’s teaching 
1. Use of student questionnaire, data — specify 


NO Gh © D— 


. Use of self-evaluation data — specify 
. Use of peer ratings 
. Use of administrative ratings 
. Use of collegial teaching situations In order to discuss 
teaching 
. Participates In seminars, workshops, or seminars almed 
at Improving teaching. 7 
. Reads literature on teaching and learning and Its im- 
provement 
. Critically reviews new teaching materials for possible 
application 
9. Pursues line of research that contributes to teaching 
10. Attends seminars or workshops to update knowledge of 
developments in one's discipline 
11. Authors text or other instructional materiais 
12. Attempts Instructional innovations 
13. Evaluates instructional materials 
14. Attends professional meetings likely to Influence 
teaching 
15. Knows about and uses teaching support services (e. Ge, 
library, student counselling services, teaching/learning 
centre) to improve teaching and learning. 
16. Edits professional journal related to teaching (@.g., 
Journal of Chemical Education) 
17. Participates In new course development and existirg 
course revision activities where needed ; 
18. Consults with colleagues re: course improvement 
19. Does action research on his/her own teaching 
20. Participates in curriculum committees 
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Reports from Others. 


A. Student responses 
1. Formative (while course Is in progress) questionna re 
data — what and how used? 
2. Summative (at conclusion of course) questionnaire a“ 
— what and how used? 
. Student interviews — what and how used? 
. Creates a student committee to evaluate course 
. Elected master teacher of year by students _ 
. Elected “come back” teacher of year by students 
. Open ended evaluation by students (unstructured) 
. Student reported satisfaction with consulting at otter 
out of class contacts 
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8. Colleague responses 

1. Statement of referees whc observed teaching 

. Statement of participants In team teaching 

. “Grapevine” comments of those who teach sections of 
the same course 
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4. Colleague evaluation of one’s contribution to teaching 
Improvement and course development 

5. Colleague’s ratings, from other institutions 

6. Colleagues elect one to committee on teaching. 

C. Reports from others 

1. Administrator's ratings — same institution 

2. Administrator's ratings — from other institutions 

3. Alumni ratings F 

4. Reports from parents of students 

5. Invitations to teach by outside agencies, other 
unlversitles or colleges 

6. Invitations to contribute to teaching literature 

7. Invitations to speak about teaching. 


©. Products of student attalnment 


1. Scores of students on teacher-made tests (especially 
pre and post course comparisons) 

. Student scores on standardized tests (especially pre and 
post course comparisons) 

. Student worksheets, item sets and lab manuals | 

. Student projects, essays, fleld work, etc. : 

. Publications of students on course-related work ' 

. Evidence of student attitudes toward the subject 
(especially pre and post course comparisons) 

. Student career choices influenced 

. Proportions of students who select and succeed in ; 
advanced courses In the area and credit instructor for 
this decision 

9. Proportions of students who are able to and who elect 

another course with the same professor. 
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Some principles to take into consideration 
when putting together a dossier would 
include using a variety of information in 
order to allow others to make sound judge- 
ments about commitment to teaching. A 
single statistic; e.g. student questionnaire 
drawn from a specific point in time does not 
suffice as an indicator of good teaching. 
Why? Because if one compares professor A 
with Professor B at point 1, A may be better 
than B. But suppose at point 2, (one year 
later) we note that A is losing ground and 

B is improving. A fixed time, simple cri- 
terion evaluation would punish professor B 
who is committed to improving and reward 
professor A who may not be maintaining his 
teaching effectiveness. The dossier should 
include a personal definition of good 
teaching which provides rationale for the 
information one chooses to gather. 


While preparing a dossier can be done alone, 
it is-»est done in consultation with others, 
perhaps the departmental chairperson, another 
colleague or a teaching resource person. 


Adapted from Berquist, Willian H. and Phillips, 
Steven R., "A Handbook of Faculty Development" 
Volume 2. And Ontario Universities Program 
for Instructional Development "Newsletter" 
April/May 1978. 


AND NOW FOR SOMETHING COMPLETELY DIFFERENT: 


SCORING THE COMMITTEE GAME * 
BY: Halas L. Jackim 


There are three kinds of people in the 

world - those who make things happen, those 
who simply watch things happen, and those who 
plan the Committee Game. The game is not 
simple; it requires skill and subtle strategies 
not unlike poker. It has taken the author 23 
years of infiltration in committees to enable 
him to distill and reveal the real nature of 
the game. Most committee-game devotees become 
compulsive, insisting that their committees 
meet no less than, once a week, while real 


‘connoisseurs of the art are often found 


holding breakfast meetings in order to squeeze 
in that extra game or two each week. 


Before describing the rules, etiquette, and 
scoring system for the game, allow me to state 
the overall objectives: a) To prevent the de- 
velopment of significant decisions; b) To 
delay or diffuse an issue by making it un- 
recognizable. 


ETIQUETTE 


It is proper to show disgust for any act 

aimed at achieving a speedy discharge of a co- 
mmittee's mission. Use of the words ‘dirty', 
‘mechanistic’, 'structured', and ‘the cult of 
efficiency’ is in good taste when describing 
such acts. Resist all attempts at delegation 
of personal responsibility; insist on com- 
mittee action. 


RULES 


Any number may play, although a group which 
numbers five to fifteen members provides the 
greatest satisfaction. Players start playing 
as individuals, but team up with compatibel 
counterparts as the game progresses. 


SCORING 


The true pleasure in serving on committees 
comes from scoring and watching other scores. 
The following should help the novice keep up 
with the more experienced player and the ex- 
perienced player to better assess his game: 


- 5 points for converting the obvious meaning 
of a statement into something different by 
approaching it from a creative-oblique angle. 

- 5 points for ‘forgetting’ an earlier decision 
and bringing it up for renewed discussion 
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(additional 5 points if player gets away 
With LE) « 

- 5 points for citing a study, book or 
other authority and quickly changing the 
subject before being questioned about it. 
- 5 points for criticizing the adminis- 
tration of the institution. 

- 5 points for canonizing the need for 
student participation. 

- 5 points for gesticulating with glasses, 
pipe, or pen to dramatize a point. 

- 5 points for telling the group that it 
has a communication problem. 

- 5 points for delaying a decision by 
using ploy of need to consult with con- 
stituency. 

- 5 points for taking twice as long as 
needed to say something. 

- 10 points for injecting an irrelevant 
issue (5 extra points if it manages to 
sidetrack the committee's deliberations.) 
- 10 points for getting committee to re- 
consider its procedures. 

- 10 points for throwing in a new tern, 
preferable from a foreign language. 

- 15 points for phrasing a statement in 
such a way that a player opposed to it 
fails to recognize it and votes for it. 
- 25 points for wearing down the opposi- 
tion to the point where it will vote for 
anything simply to adjourn the meeting. 


A final warning is in order. Players can 
be disqualified for: 


- Persisting in their desire to clarify 
committee's goals. 

- Insisting on identifying obstacles to 
committee's progress. 

- Expecting to see implementation of 
committee's recommendations. 


*Taken from the ‘Chronicle of Higher 
Education' July 21st, 1975. 


ANNOUNCEMENTS : 


Workshop on ‘Student Motivation' 
December 6th and 7th at Lacolle. 
For registration call Noreen 
Linton, local 397 or 695. 


INDIVIDUAL EVALUATIONS! 


We are prepared to design and 
process course evaluation 
questionnaires for any faculty 
who request it. If you would 
like to use this service, please 


contact us soon. Local 397 or 695. 


